Emotion Recognition Systems At Workplace Threaten Privacy Rights

Emotion recognition technologies also endanger fundamental rights without proper justification. They have been found to discriminate on the basis of race, gender and disability

Can artificial intelligence (AI) tell whether you're happy, sad, angry or frustrated? According to technology companies that offer AI-enabled emotion recognition software, the answer to this question is yes.

But this claim does not stack up against mounting scientific evidence. What's more, emotion recognition technology poses a range of legal and societal risks – especially when deployed in the workplace. It is for these reasons that the European Union's AI Act, which came into force in August, bans AI systems used to infer emotions of a person in the workplace – except for “medical” or “safety” reasons. In Australia, however, there is no specific regulation of these systems. As I argued in my submission to the Australian government in its most recent round of consultations about high-risk AI systems, this urgently needs to change.

A new and growing wave:

The global market for AI-based emotion recognition systems is growing. It was valued at $34 billion in 2022 and is expected to reach $62 billion by 2027. These technologies work by making predictions about a person's emotional state from biometric data, such as their heart rate, skin moisture, voice tone, gestures or facial expressions.

Next year, Australian tech startup inTruth Technologies plans to launch a wrist-worn device that it claims can track a wearer's emotions in real time via their heart rate and other physiological metrics. inTruth Technologies founder Nicole Gibson has said this technology can be used by employers to monitor a team's “performance and energy” or their mental health to predict issues such as post-traumatic stress disorder.

She has also said that inTruth can be an “AI emotion coach that knows everything about you, including what you're feeling and why you're feeling it”.

Emotion recognition technologies:

There is little data about deployment of emotion recognition technologies in Australian workplaces. However, we do know that some companies used a video interviewing system offered by a US-based company called HireVue that incorporated face-based emotion analysis. This system used facial movements and expressions to assess the suitability of job applicants. For example, applicants were assessed on whether they expressed excitement or how they responded to an angry customer.

HireVue removed emotion analysis from its systems in 2021 following a formal complaint in the United States. Emotion recognition may be on the rise again as Australian employers embrace artificial intelligence-driven workplace surveillance technologies.

Lack of scientific validity:

Companies such as inTruth claim emotion recognition systems are objective and rooted in scientific methods. However, scholars have raised concerns that these systems involve a return to the discredited fields of phrenology and physiognomy. That is, the use of a person's physical or behavioural characteristics to determine their abilities and character. Emotion recognition technologies are heavily reliant on theories which claim inner emotions are measurable and universally expressed. However, recent evidence shows that how people communicate emotions varies widely across cultures, contexts and individuals. In 2019, for example, a group of experts concluded there are “no objective measures, either singly or as a pattern, that reliably and uniquely” identify emotional categories.

For example, someone's skin moisture might go up, down or stay the same when they are angry. In a statement, inTruth Technologies founder Nicole Gibson said “it is true that emotion recognition technologies faced significant challenges in the past”, but that “the landscape has changed significantly in recent years”.

Infringement of fundamental rights:

Emotion recognition technologies also endanger fundamental rights without proper justification. They have been found to discriminate on the basis of race, gender and disability. In one case, an emotion recognition system read black faces as angrier than white faces, even when both were smiling to the same degree. These technologies may also be less accurate for people from demographic groups not represented in the training data.

Gibson acknowledged concerns about bias in emotion recognition technologies. But she added that “bias is not inherent to the technology itself but rather to the data sets used to train these systems”.

As a surveillance tool, emotion recognition systems in the workplace pose serious threats to privacy rights. Such rights may be violated if sensitive information is collected without an employee's knowledge. There will also be a failure to respect privacy rights if the collection of such data is not “reasonably necessary” or by “fair means”. Workers' views:

A survey published earlier this year found that only 12.9 per cent of Australian adults support face-based emotion recognition technologies in the workplace. The researchers concluded that respondents viewed facial analysis as invasive and the technology as unethical and highly prone to error and bias.

In a US study also published this year, workers expressed concern that emotion recognition systems would harm their wellbeing and impact work performance. They feared that inaccuracies could create false impressions about them. In turn, these false impressions might prevent promotions and pay rises or dismissals.

(The writer is a researcher and lawyer at La Trobe University)